Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable ci-integration tests on GHA (incl fixing a few tests) #2

Closed
wants to merge 12 commits into from

Conversation

cognifloyd
Copy link
Member

@cognifloyd cognifloyd commented Mar 23, 2021

Move integration tests to GHA

We have to fix a few tests to make the transition:

  • improve CI output
  • improve GHA folding
  • fix virtualenv lock files issue again
  • Switch tests to check w/ CA that is present in GHA
  • apt download cache in GHA
  • use bitnami/rabbitmq service container in GHA
  • bust python cache
  • Use virtualenv from st2 virtualenv by default
  • Install virtualenv in --user on github
  • Add workaround for failing stdin test
  • fix check-dependency-conflicts by updating pip-tools

cognifloyd and others added 12 commits March 22, 2021 19:22
- remove git clone progress reporting (reduce CI noise)
- report current user
- no random ports
- use separate rabbitmq.conf file on github
- ci: temporarily disable all but ci-integration
- ci: cp ssl_certs into rabbitmq container
- reenable management plugin
- manual custom.conf
- add python-version for ci-integration tests
- add default rabbitmq user/pass
- do not hardcode RMQ user/pass in tests
- override more rabbitmq test urls
- fix integration test conf files on the fly
- go back to guest:guest with rabbitmq
- finish reverting rmq guest:guest stuff
- Reconfigure RabbitMQ for ci-unit tests as well
If a virtualenv was built with symlinks instead of copies of the python
binary, as happens by default on GHA integration tests, then we end up
using the system virtualenv binary instead of what we installed.

Currently GHA has the same version of virtualenv, but we build
virtualenvs with --system-site-packages, and we need it to use the st2
virtualenv as the "system" dir instead of the GHA provided one. So,
don't get the realpath of the python binary.

Also, fall back to system virtualenv bin for backwards compat.
- install virtualenv in --user
- pip uninstall --user
- Try to recreate failure on Travis
- leave TODO/FIXME around broken test
- Use close_fds=True.
- add workaround for failing test
When we updated pip, we missed updating pip-tools to support the newer
pip version, so there were errors (that didn't fail the build, oops)
like these (fixed in pip-tools 5.4, pip-tools-PR-1216:

virtualenv/bin/pip-compile req.txt

Traceback (most recent call last):
  File "virtualenv/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/piptools/scripts/compile.py", line 458, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/piptools/resolver.py", line 173, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/piptools/resolver.py", line 278, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/piptools/resolver.py", line 388, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/piptools/repositories/local.py", line 75, in get_dependencies
    return self.repository.get_dependencies(ireq)
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/piptools/repositories/pypi.py", line 232, in get_dependencies
    download_dir, ireq, wheel_cache
  File "/home/runner/work/st2/st2/virtualenv/lib/python3.6/site-packages/piptools/repositories/pypi.py", line 163, in resolve_reqs
    wheel_download_dir=self._wheel_download_dir,
TypeError: make_requirement_preparer() got an unexpected keyword argument 'wheel_download_dir'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants